The invention described herein was made by employee of the United States Government. The invention may be manufactured and used by or for the governmental purposes without the payment of royalties thereon or therefor.
The present invention relates generally to image processing, and particularly to image enhancement following JPEG image compression and decompression. Prior methods for enhancing compressed images are beset with disadvantages or limitations. To achieve acceptable compression rates, the JPEG standard adopted a lossy compression technique. Furthermore, the JPEG, MPEG, H.261 and HDITV image and video coding algorithms are severely impaired by blocking artifacts when operating at low bit rates. A compressed image becomes distorted when compressed by lossy methods. Distortion in the image can be confined to hidden areas of the image by carefully implementing the JPEG standard and several enhancement methods for reducing the distortion have been attempted. However, in some regions of the image these prior methods may further degrade the image. Prior methods for reducing distortion introduced by the JPEG compression algorithm can introduce a high-frequency artifact around high contrast edges. Algorithms for removing artifacts exist, but can mistakenly remove thin lines and smooth over important texture. Errors also arise between adjacent 8xc3x978 pixel image blocks created by opposite-signed round-off errors in the two blocks. One way to remove these errors is to filter the image using a low-pass filter. However, other image regions that contain important high frequency information may suffer a loss in fidelity. Likewise, many other post-processing algorithms are most effective when they are selectively applied only to certain regions of an image. Previous techniques are mainly non-linear filtering methods based on local pixel statistics rather than on local frequency content. Prior efforts include; JPEG Cross Block Smoothing (CBS), Projection Onto Complex Sets (POCS), and a variety of nonlinear filters. These offer lower peak signal to noise ratio (PSNR) and lower subjective quality than the improved algorithms proposed in the present invention, however. Additional background information may be found in the following literature references:
IJG JPEG Software 0 1990-95, Tom G. Lane.
Canny Edge Detector from xe2x80x9cX-based Image Processing Tools and Environment (XITE)xe2x80x9d, S. Boe, 1994.
JPEG image-adaptive DCT coefficient quantization software xe2x80x9cadaptQ( )xe2x80x9d as described and made available by M. Crouse and K. Ramchandran, xe2x80x9cJoint Thresholding and Quantizer Selection for Transform Image Coding: Entropy-Constrained Analysis and Applications to Baseline JPEG,xe2x80x9d IEEE Transactions on Image Processing, vol. 6, No. 2. pp. 205-297, February 1997.
xe2x80x9cTheory and Applications of the Estimated Spectrum Adaptive Post-filterxe2x80x9d, Ph.D. thesis, Georgia Institute of Technology. (copyright) 1998 Irving Linares, Pages 1-113 May, 1998.
ESAP and IPF algorithms of the present invention significantly reduce the blocking artifacts resulting from high compression by reusing the DCT coefficient local frequency characteristics to control a pixel-adaptive non-linear post-filter or a pre-post filtering system. This allows more compression and better quality compression when compared with the default JPEG compression parameters. These algorithms provide a new frequency-based pixel-adaptive filtering algorithm which may be used to enhance de-compressed JPEG images and to enhance MPEG sequences for Internet or HDTV video applications.
One object of the present invention is to reduce blocking artifacts in low-bit-rate JPEG images, particularly in systems wherein image pixels are coded using bit to pixel ratios less than or equal to 0.25 bits per pixel.
Another object is to improve PSNR and perceptual quality of coded images simultaneously and to demonstrate the feasibility of extending still image pre- and post-processing concepts to MPEG and HDTV standards.
A further object is to suggest use of ESAP/IPF algorithms to improve very low rate MPEG video sequences such as those processed by popular 28.8 k-56 k modems generally used with web browser streaming video viewers such as RealPlayer(trademark) or Streamworks, for example.
ESAP relies on DFT analysis of the DCT and is compliant with the coded stream syntax of the Independent JPEG Group (IJG) Version 5b Software. At the decoder, ESAP estimates the 2-D pixel-adaptive bandwidths directly from the dequantized DCT coefficient. These coefficients are used to control a 2-D spatially adaptive non-linear post-filter. The algorithm optionally performs directional filtering parallel to the edges with no filtering across the edges. Post-filtering images show minimal blurring of their true edges while blocking is significantly removed.
IPF is based on the concepts of dbx audio noise reduction. IPF is a pre-post filtering system that uses inverse pair 2D filters for high frequency pre-emphasis before encoding and high frequency de-emphasis after decoding. Convergence to a unique minimum mean square error (MMSE) is possible for fixed quantization matrices. However, convergence can not be guaranteed when image-adaptive DCT quantization is jointly optimized under pre-post filtering.
The ESAP methods of the present invention have been successfully applied to JPEG color images, synthetic aperture radar (SAR) images, and image sequences. Typical PSNR improvement depends on the image, the encoding method, and the bit rate. For 512xc3x97512 8-BPP gray-scale images improvement in the range between 0.5-3.2 dB over baseline JPEG has been observed. Adaptive quantization has been observed to improve 5.6 dB for 1008xc3x971008 8-BPS SAR images at 4-BPP over baseline JPEG. A comparison of all the treated techniques is presented at the conclusion of thesis document (Irving Linares, Theory and Applications of the Estimated Spectrum Adaptive Post-filter, Georgia Institute of Technology. (copyright) 1998 Irving Linares, Pages 1-113 May, 1998) incorporated herein by reference. ESAP and IPF algorithms of the present invention significantly reduce blocking artifacts resulting from high compression by reusing the DCT coefficient local frequency characteristics to control a pixel-adaptive non-linear post-filter or a pre-post filtering system. This allows more compression and/or better quality compression when compared with the default JPEG compression parameters alone. The subjective quality of image improvement is more clearly shown by means of foregoing software-simulated resultsxe2x80x94including several image samples. The ESAP and IPF algorithms may also improve the visual quality of DCT coded images, such as JPEG images and may be extended to MPEG video sequences, since MPEG is largely a DOT coder.